Network intrusion detection systems (IDS) have historically relied on concepts such as signature matching, protocol analysis, and techniques with predefined thresholds such as for number of packets per time interval. A drawback of traditional signature matching is that if no signatures exist to help identify a particular threat, that threat can evade detection. Protocol analysis works by understanding how assorted protocols should work, and verifying that traffic conforms to the behavior expected of that protocol. Protocol analysis, however, requires extensive processing and knowledge of the protocols being used.
As the volume and complexity of network traffic has increased, the amount of raw network data has expanded to the point where it can be difficult to identify events of interest, e.g., network traffic anomalies that might be associated with a security threat, by applying thresholds to raw network data. Basic visualization tools have been developed as means to help administrators recognize anomalies in traffic, such as by representing different types of network traffic in different colors on a graph. However, such tools require human monitoring and human intelligence to examine a visualization to discern what traffic might pose a threat and, as in other approaches to processing raw data, they do not afford a reliable or timely way of identifying traffic anomalies such as may be associated with a security threat.
Therefore, a better way of evaluating network data for threats is needed.